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ABSTRACT 

The  Objective  Force  concept  requires  technology 
advantage  over  the  adversaries  to  design,  develop,  model 
and  simulate  weapon  systems  that  are  strategically 
responsive,  deployable,  agile,  versatile,  lethal  and 
survivable.  As  the  Army  transforms  there  is  a  renewed 
interest  in  the  concept  designs  of  new  projectile-target 
weapons  systems.  As  the  modular  systems  are  developed, 
optimized  and  ultimately  integrated  to  the  continuously 
evolving  FCS,  there  is  a  critical  need  for  a  “next- 
generation”  sate-of-the-art  computer  modeling  and 
simulation  tools.  In  addition,  there  is  also  critical  need  for 
have  a  single,  validated,  maintainable  high  performance 
computing  (HPC)  code  which  captures  the  physics  of 
short  and  term  dynamic  events  to  drastically  reduce 
simulation  times.  In  this  paper,  we  present  a  new  and 
advanced  HPC  based  multi-scale  in  time  projectile-target 
simulation  environment  for  lethality  and  survivability  that 
is  useful  for  FCS  design  concepts.  Illustrative  examples 
are  considered  that  demonstrate  the  capabilities  of  the 
proposed  approach. 


1.  INTRODUCTION 

Strategically  agile  and  responsiveness  of  Armored 
vehicles  depends  on  the  sensitive  electronic  components. 
A  structural  design  that  eliminates  potential  failure  of 
these  sensitive  electronic  components  is  influenced  by  the 
shock  wave  propagating  through  the  structure  due  to 
impact  loads  and  is  highly  critical.  Numerous  efforts 
during  the  past  several  years  have  focused  on  accurate 
modeling  and  predicting  the  force-time  histories  of  local 
events  such  as  contact,  impact,  penetration,  and 
penetration  induced  damage  due  to  projectiles  to 
accurately  predict  the  global  structure  response  of  the 
target.  See  for  example,  work  by  Saliba  et.  al,  1996  and 
references  there  in. 

In  lieu  of  this,  there  is  a  renewed  interest  in  the  Army 
for  the  next  generation  of  projectile-target  interaction 
software  technology  that  can  accurately  predict  the  global 
structure  response  of  the  target  due  to  local  events  with 
drastic  reduction  in  the  simulation  times.  These 
simulations  in  general  are  a  multi-scale  in  time  physics 
where  events  are  occurring  with  respect  to  different  time 
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Figure  1.  Illustration  of  the  multi-scale  in  time  events  in  a 
projectile-target  interaction  system.  A  unified  scalable 
validated  HPC  based  multi-scale  in  time  software  is  highly 
desirable  to  account  for  proper  data  transfer  of  physics 
from  explicit  to  implicit  and  implicit  to  explicit  phases  of 
the  computations. 


scales.  For  example,  the  events  occurring  at  small  time 
scales  (micro  time  scales),  namely,  are  the  short-term 
localized  penetration/impacts  of  projectiles  into  the 
targets.  Similarly  events  occurring  at  larger  time  scales 
(tenths  of  a  second  or  even  longer)  are  the  global  dynamic 
response  such  as  structural  vibrations  of  the  targets.  When 
subjected  to  short-term  contact-impact-penetration 
followed  by  the  post  dynamics  of  the  long-term  contact- 
impact  effects  they  are  inherently  all  coupled  in  the 
problem  definition.  For  example,  low  frequency  and  long¬ 
term  dynamics  are  dominant  from  the  initiation  of  flight 
dynamic  conditions  till  contact  is  made  and  subsequently 
after  the  weapon  leaves  the  target.  Similarly,  high 
frequency  and  short-term  dynamics  are  dominant  during 
the  contact-impact-penetration  stage.  These  sequences  of 
events  are  described  in  Fig.  1.  Therefore,  there  is  also 
critical  need  for  have  a  single,  validated,  maintainable 
high  performance  computing  (HPC)  code  which  captures 
the  physics  of  short  and  term  dynamic  events  to 
drastically  reduce  simulation  times.  In  addition,  there  is 
wide  range  of  applications  in  which  distinct  regimes  of 
explicit  and  implicit  computations  can  be  identified.  To 
cite  a  few  readers  are  referred  to  Finn  et  al.  1995  and 
Narkeeran  and  Lovell,  1996. 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

00  DEC  2004 

2.  REPORT  TYPE 

N/A 

3.  DATES  COVERED 

4.  TITLE  AND  SUBTITLE 

5a.  CONTRACT  NUMBER 

Multi-Scale  In  Time  Projectile-Target  Hpc  Simulations  For  Lethality 

5b.  GRANT  NUMBER 

/A11U  OUI  Ylv ilUilllj 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Department  of  Mechanical  Engineering,  University  of  Minnesota  Army 
High  Performance  Computing  Research  Center,  Minneapolis,  MN  55455 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release,  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

See  also  ADM001736,  Proceedings  for  the  Army  Science  Conference  (24th)  Held  on  29  November  -  2 
December  2005  in  Orlando,  Florida. ,  The  original  document  contains  color  images. 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION  OF 
ABSTRACT 

uu 

18.  NUMBER 
OF  PAGES 

8 

19a.  NAME  OF 
RESPONSIBLE  PERSON 

a.  REPORT 

unclassified 

b.  ABSTRACT 

unclassified 

c.  THIS  PAGE 

unclassified 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


For  these  multi-scales  in  time  physics  simulations, 
most  of  the  researchers  are  relying  on  a  standalone 
explicit  computational  technology  such  as  that  in 
PARADYN  (Hoover  et  al.,  1994),  EPIC  (Johnson,  et  al., 
1997),  etc.,  for  the  entire  transient  analysis.  In  these 
intensive  computation-based  simulations,  the  conditional 
stability  associated  with  the  Linear  Multi  Step  based 
explicit  time  integration  technique  such  as  the  central 
difference  method  severely  limits  the  total  simulation 
duration  to  be  completed  in  a  reasonably  practical  amount 
of  time.  More  specifically,  the  post  dynamics  of  contact- 
impact  dynamic  simulation  is  unbearable  and  is  very 
expensive  when  employing  these  standalone  explicit 
computations.  Alternatively,  the  implicit  computational 
techniques  that  are  employed  in  ABAQUS,  2004; 
NIKE3D,  1995,  etc.,  are  not  suitable  in  the  high  velocity 
impact  simulations  regime  since  it  cannot  capture  the 
stress  wave  propagation  at  larger  time  step  sizes. 
Alternatively,  during  the  post  dynamics  simulation 
regime,  the  parallel  implicit  computations  are  not  scalable 
because  of  lack  of  a  scalable  solver  technology  that  is 
required  to  effectively  solve  implicit  system  of  equations 
at  each  time  step  and  during  the  nonlinear  iterations 
phase. 

We  have  developed  and  herein  present  a  HPC  based 
multi-scale  in  time  physics  software  based  on:  1)  novel 
computational  algorithms  with  a  smart  switch  which 
adaptively  decide  explicit  and  semi-explicit  strategies,  2) 
robust  contact-impact  algorithms  describing  the  physics 
of  contact  via  linear  complementary  formulations  for 
normal  direction  and  augmented  Lagrangian  formulation 
in  tangential  direction,  and  3)  large  strain  finite 
deformation  mechanics  with  a  new  arbitrary  reference 
configuration  methodology. 

The  research  efforts  are  focused  on  delivering  this 
HPC  based  end  product  for  the  Army  initiatives  and  to 
successfully  demonstrate  the  effectiveness  to  the 
Objective  force.  The  benefits  of  this  HPC  software 
technology  are  many  folds.  They  are:  1)  reduced  run 
times  of  an  order  of  magnitude  for  problems  involving 
contact-impact  and  long-term  post  dynamics  as  against 
employing  standalone  explicit  computation  based  codes, 
2)  a  unified  code  instead  of  a  combination  of  codes  for 
avoiding  erroneous  approximations  introduced  in 
combining  different  codes  and  for  ease  of  maintainability, 
and  3)  better  parallel  scalability  which  implies  effective 
utilization  of  the  Army  HPC  resources.  All  these 
technological  advances  and  the  potential  impact  are 
demonstrated  in  this  paper. 

2.  FORMULATION 

The  projectile-target  interaction  involves  a  multitude 
and  multiple  physics  such  as  the  rigid  body  motion,  the 


structural  vibrations,  the  contact/impact  events,  the 
damage  and  fracture  process  including  non-linear  material 
and  large  deformation.  In  this  section,  the  governing 
equations  of  these  aspects  are  briefly  described. 

Momentum  equation:  The  weakform  of  the  momentum 
equation  can  be  stated  as 

at 

=  (w,b)a  +(w,f)TfW  +  (<%,cr„  +o-r)rc(0 

where  w  is  the  virtual  displacement  field,  v  is  the  velocity 
field,  p  is  the  density,  r|  is  the  viscous  damping,  D  is  the 
virtual  velocity  strain,  u  is  the  Cauchy  stress,  b  is  the 
body  force,  /  is  the  external  force,  g  is  the  gap  vector 
function,  ern  is  the  normal  contact  force,  uT  is  the  frictional 
force,  Sg  is  the  virtual  gap,  Q,  is  the  material  domain,  r/t) 
is  the  Neumann  boundary,  rc(t)  is  the  contact  boundary, 
and  (•,•)  represents  the  inner  product. 

Contact  constraints  -  Impenetrability  conditions:  The 

weakform  associated  with  contact  constraints  that  lead  to 
complementary  conditions  can  be  stated  as 

(T*>g„  -Kn)rAt)  =° 

,t;,>o 

>*>,,(0  =  0 

where  t„  is  the  virtual  normal  contact  force,  g„  is  the 
normal  direction  gap  vector,  A.c„  is  the  Lagrangian 
multiplier  represents  the  normal  contact  force,  and  it  is  the 
normal  contact  direction. 

Frictional  boundary  condition:  Similarly,  the  weakform 
associated  with  frictional  force  at  the  contact  surface  can 
be  stated  as 

(rT,gT  - 00)^1  =  o 

y/T(aT)>0 

ZT>0 

OMO  =  o 

where  rT  is  the  tangential  virtual  frictional  force,  gT  is  the 
tangential  direction  gap  vector,  Acr  is  the  Lagrangian 
multiplier  represents  the  frictional  force,  and  is  the 

friction  law. 


Constitutive  Equation:  The  hypoelastic  constitutive 
equation  are  considered  which  can  be  written  in  the  form 


a  =  (l-Dd)1  :a  =  CT:(l-Dd)1 


(7  =  C:(D-y 


8F(cr,Dd,Y) 


dcr 
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where  cr  is  the  effective-Cauchy-stress,  Dd  is  the  damage 
tensor,  crv  is  the  objective  effective-Cauchy-stress,  y  is 
the  Lagrangian  multiplier  represents  the  plastic  and 
damage  flow  magnitude,  F(cr;Dd,Y)  is  the  plastic  and 

damage  dissipative  potential  function,  and  Y  is  the  energy 
release  function  due  to  damage. 


is  subject  to  no  external  force,  thus  the  supremum  of  the 
acceleration  is  zero.  And,  also  the  collision  between  the 
bodies  has  not  yet  taken  place,  which  is  characterized  by 
the  gap  distance  between  the  projectile  and  the  target  to 
be  greater  than  a  given  critical  value,  gcrit.  Therefore,  we 
have 

I=I0  {amax  =  sup  |a|=0}  u  {gmi„=inf|g|>gCrit}  (2) 

For  rigid  body  motion  (except  the  rigid  rotation),  no 
physical  models  and  time  integration  is  needed,  and  the 
system  can  simply  update  the  displacement  by 
multiplying  the  velocity  by  the  time  step  size. 

3.2  Contact/Impaet 


3.  ADAPTIVE  SMART  SWITCH  ALGORITHM 

An  adaptive  algorithm  based  on  a  smart  switch 
formulation  was  developed,  implemented,  and  tested  for 
various  applications  involving  contact-impact  events.  This 
algorithm  switches  between  the  explicit  computations  and 
semi-implicit  computations  based  on  the  detection  of  the 
physical  event.  For  explicit  computations,  a  forward 
displacement  central  difference  (FDCD)  algorithm  (Sha  et 
ah,  1996)  was  employed  and  for  the  semi-implicit 
computations  a  nonlinearly  explicit  and  L-stable  (EL) 
algorithm  (Zhou  et  ah,  2004)  was  employed.  The 
physical  events  can  be  rigid-body  in-flight  motion  of  the 
projectile,  low/high  velocity  impact,  penetration  and 
damage,  and  post-impact  structural  dynamic  vibrations. 
This  formulation  is  particularly  useful  if  these  physical 
events  are  in  a  sequence  as  the  time  step  size  employed 
during  one  phase  of  computations  to  another  is  global  in 
nature. 

The  physical  events  that  are  associated  with  the 
projectile-target  interaction  can  be  classified  based  upon 
the  following  four  parameters.  They  are  the  acceleration 
of  the  nodes,  gap  function  between  the  colliding  bodies, 
damage  stress,  and  the  damage  state.  Then,  if  I  is  the 
event  function  which  detects  these  events,  it  can  be  stated 
as: 

I  (Io,I„I2,l3,l4)  (  Alarm.  ^'Aril)  (1) 

where  I  represents  the  physical  events,  alll;lx  is  supremum 
of  the  acceleration  vector,  a,  gm;n  is  the  infimum  of  the 
gap  vector,  g,  <jd  is  the  damage  stress  threshold  value,  and 
(ocrit  is  the  critical  damage  state.  The  evaluation  criteria  to 
determine  the  specific  physical  events  is  described  next. 

3.1  In-Flight  Motion 

In-flight  motion  predominantly  consists  of  rigid  body 
motion.  For  the  rigid  body  motion,  we  assume  the  system 


When  the  minimum  distance  between  the  projectile 
and  the  target  is  smaller  than  the  given  critical  value,  and 
the  maximum  second  invariant  of  the  stress  is  less  than 
the  material  damage  stress  threshold  value,  we  consider 
the  contact/impact  events  are  taking  place  in  the  system. 
This  condition  can  be  stated  as  follows. 

I=Ii  {amax>0}u{gmin<gcrit}u{a2<ad}  (3) 

During  this  contact/impact  stage,  the  code  adaptively 
switches  to  explicit  computations  for  solving  equations 
pertaining  to  structural  dynamics,  finite  deformation  and 
contact  constraints. 

3.3  Damage 

When  the  minimum  distance  between  the  projectile 
and  the  target  is  less  than  the  given  critical  value  and  the 
maximum  second  invariant  of  the  stress  is  greater  than  the 
damage  stress  threshold  value  but  the  scalar  damage 
parameter  is  less  than  the  critical  damage  state,  we 
consider  the  system  is  undergoing  the  damage  stage.  This 
state  is  given  by, 

I=I2  {amax>0}u{gmin<gcrit}u{a2>ad}u{aX(ocrit}  (4) 

In  this  stage,  the  code  still  employs  explicit 
computations  in  solving  equations  pertaining  to  structural 
dynamics,  finite  deformation,  contact  constraints,  and  the 
damage  mechanics. 

3.4  Penetration 

When  the  minimum  distance  between  the  projectile 
and  the  target  is  less  than  the  given  critical  value,  the 
maximum  second  invariant  of  the  stress  is  greater  than  the 
material  damage  stress  threshold  value  but  the  scalar 
damage  parameter  is  greater  than  the  critical  damage 
state,  we  consider  the  system  is  undergoing  the 
penetration  stage.  This  is  given  by, 


I=I3  <-  {amax>0}u{gmin<gcrit}u{a2>Cd}u{(B>CBcrit}  (5) 

In  the  penetration  stage,  the  code  still  employs 
explicit  computations  in  solving  equations  pertaining  to 
structural  dynamics,  finite  deformation,  contact 
constraints,  and  damage  mechanics  with  erosion. 

3.5  Post-Impact/Penetration  Dynamics 

When  the  supremum  of  the  acceleration  is  not  equal 
to  zero,  but  the  minimum  distance  between  the  projectile, 
the  target,  and  the  debris  due  to  the  penetration  is  greater 
than  the  given  critical  value,  we  consider  the  system  is 
undergoing  post-impact/penetration  dynamics  stage.  This 
is  given  by, 

I=I4  {amax>0}u{gmin>gcrit}  (6) 

In  the  post-impact/penetration  dynamics,  the 
computations  adaptively  switch  to  semi-implicit 
computations  with  a  larger  time  step  size  for  the  solution 
of  equations  pertaining  to  structural  dynamics  and  finite 
deformation. 

In  summary,  the  projectile-target  interaction  involves 
multiple  physical  events  and  some  of  the  physical  events 
are  low-frequency  dominated  (long  time  scale)  and  some 
of  the  physical  events  are  high-frequency  dominated 
(short  time  scale).  The  design  of  the  current  smart  switch 
adaptive  algorithm  is  based  on  the  assumption  that  these 
physical  events  associated  with  the  projectile-target 
interaction  occur  sequentially  wherein  during  each  stage 
of  the  physical  event  we  employ  the  appropriate  physical 
models  and  time  integration  methods  to  handle  the 
specific  physical  events.  Efforts  are  currently  underway  to 
adaptively  handle  multiple  physical  events  based  on 
element  based  computations  with  smart  switch  where 
some  elements  solely  explicit  while  some  elements  are 
solely  semi-implicit  in  the  same  finite  element  mesh. 

4.  PARALLEL  FORMULATIONS 

The  increasing  complexity  of  this  emerging  and  next 
generation  survivability-lethality  technology  needs  higher 
fidelity  in  numerical  models  to  capture  the  geometry  and 
material  behavior  under  realistic  operational  loading 
scenarios,  for  large  scale  geometries  such  as  an  entire 
army  vehicle  to  be  included  in  these  simulations  (see  Fig. 
4).  This  is  unlike  existing  approaches  where  only  a 
representative  section  of  the  vehicle  is  mostly  considered 
(for  example  a  plate  section)  for  the  simulation  in 
survivability-lethality  designs.  Increasing  model  fidelity 
requires  high  mesh  resolutions.  Longer  simulation  times 
are  needed  to  assess  post  dynamic  simulations  after 
contact,  impact,  penetration,  and  damage  events. 


Computational  requirements  grow  dramatically  as  the  size 
and  complexity  of  the  computational  models  increase.  As 
the  element  size  is  decreased  to  capture  the  finer  details 
such  as  in  large  deformations  in  geometry  and  for  overall 
response,  the  time  step  also  decreases  to  satisfy  the 
stability  criteria  in  explicit  time  integration  during  the 
small  time  scale  simulation  regime.  This  reduction  in  the 
time  step  results  in  the  need  for  long  CPU  times  to 
complete  a  simulation.  Scalable  parallel  computations  and 
development  and  implementation  of  new  computational 
methods  and  HPC  algorithms  which  solely  do  not  reliy  on 
explicit  features  but  fusion  of  explicit/semit-implicit 
technologies  are  the  primary  focus  and  an  important 
component  of  the  multi-scale  in  time  computational 
technology.  Developments  that  are  platform  independent 
and  can  be  ported  to  different  HPC  architectures,  we  use 
message-passing  interface  (MPI). 

An  efficient  parallel  finite  element  procedure  for 
contact-impact  computations  within  the  framework  of 
multi-scale  in  time  physics  is  needed.  The  scalability  of 
this  depends  on  the  following  key  kernels:  i)  parallel 
mesh  generation,  ii)  parallel  mesh  partitioning,  iii) 
parallel  motion  integration  and  solvers,  iv)  parallel  global 
contact  search,  and  v)  parallel  contact  interface  solver 
(linear  complementary  equations).  The  parallel  mesh 
generation  is  not  considered  in  this  research.  The  kernels 
such  as  parallel  motion  integration  and  solvers  that  are 
typical  of  transient  non-linear  finite  element  computations 
are  omitted  in  this  in  paper.  The  reader  is  referred  to 
Kanapady  and  Tamma,  2003,  for  more  details  for 
paralleization  aspects  pertaining  to  this  kernel.  In  this 
paper  attention  is  restricted  to  parallel  mesh  partitioning, 
parallel  contact  search  and  the  parallel  contact  interface 
solver. 

4.1  Mesh  Partitioning 

The  first  stage  of  parallization  involves  partitioning 
of  the  finite  element  mesh  into  sub-domains  as  shown  in 
Fig.  3  and  5.  Each  sub-domain  is  then  mapped  on  to  a 
processor  with  the  objective  of  balancing  the 
computational  loads  and  minimizing  the  communications 
between  all  the  processors.  Mesh  partitioning  considering 
only  the  finite  element  force  calculation  may  lead  to 
unstructured  communication  during  global  contact  search 
as  contact  segments  may  be  owned  only  by  few 
processors.  In  addition,  where  the  contact  is  going  to  take 
place  is  not  known  a-priori  and  also  this  keeps  changing 
during  the  simulation,  and  hence  load  imbalances  may 
occur.  Two  different  partitioning  stages  can  be 
considered:  one  with  finite  elements  as  the  computational 
load  and  the  second  with  the  contact  segments  as  the 
computational  load.  This  however  requires  extra 
communication  at  every  time  step  of  the  computations 
between  the  finite-element  decomposition  and  contact 
decomposition.  Multi-constraint  graph  partitioning  for 


simultaneous  data  partitioning  for  finite-element  force 
calculation  and  contact  search  can  be  employed  thereby 
eliminating  extra  communication  at  every  time  step.  To 
over  come  these  situations  a  multi-constraint  graph¬ 
partitioning  library  proved  in  ParMetis  (Karypis  and 
Kumar,  2002)  is  employed.  This  provides  the  requested 
load  balanced  partition  for  both  the  elements  and  contact 
segments  partitions  simultaneously.  The  numerical 
experiments  on  various  large-scale  finite  element  meshes 
have  shown  that  the  partitioning  time  is  reduced  by  30% 
to  40%  in  comparison  to  two  separate  partitioning 
operations.  In  addition,  this  partitioning  approach 
provides  well-balanced  partitions  for  both  finite  elements 
and  contact  segments. 

4.2  Parallel  contact  search 

Solving  for  the  contact  force  by  employing  the  linear 
complementary  approach  in  conjunction  with  the  node-to- 
segments  approach  requires  defining  the  gap  function 
constraints  and  their  corresponding  Jacobians.  The  node- 
to-segment  approach  requires  the  contact  segment  and  a 
node  pair  list  which  satisfies  the  minimum  gap  criteria. 
This  process  of  finding  contact  segment  node  pairs  list 
involves  an  all-to-all  O(n)  contact  segments  search 
leading  to  0(n2)  time.  This  can  dramatically  increase  the 
time  needed  for  the  whole  simulation.  To  reduce  cost, 
contact  searches  is  usually  performed  in  two  stages.  In  the 
first  stage,  called  global  search,  the  pairs  of  contact 
segment  and  node  that  are  potentially  be  in  contact  are 
determined.  In  the  second  stage,  called  local  search,  the 
exact  contact  segment  and  node  pairs  are  detected  from 
the  reduced  number  of  pairs  list  of  the  global  search  stage. 
Once  the  global  search  is  paralliezed  then  the  local  search 
phase  can  be  performed  independently  by  each  of  the 
processors.  In  this  paper  we  only  focus  on  the  global 
search  phase,  as  it  is  critical  for  ensuring  the  overall 
parallel  scalability  of  the  simulation. 

Zhong  and  Nilsson,  1989  have  proposed  a  sequential 
global  contact-searching  algorithm  with  contact 
hierarchies  known  as  hierarchy-territory  algorithm,  or 
HITA.  In  the  HITA  algorithm,  a  contact  system  is  defined 
which  consists  of  contact  surfaces,  contact  segments, 
contact  edges,  and  contact  nodes.  These  are  arranged  in  a 
hierarchy  to  form  a  tree.  Territories  in  terms  of  bounding 
box  are  defined  for  each  of  the  contact  objects  in  the 
hierarchies  that  define  the  domain  they  occupy  during  the 
contact  process  during  the  simulation.  If  the  territories  of 
two  contact  objects  in  certain  hierarchy  level  intersect 
then  contact  search  is  performed  between  the  contact 
objects  at  the  lower  level  else  no  contact  search  is 
performed  between  the  lower  levels.  In  our  initial 
parallelization  efforts,  we  have  parallelized  this  algorithm 
and  implemented  in  Contact-Impact-Analysis  (pCIA-3D) 
code.  The  input  this  parallel  global  contact  search  routine 
includes:  i)  is  the  entire  contact  surface  information,  ii) 


co-ordinates  and  velocities  at  the  current  time  step,  iii) 
time  step  size  and  iv)  the  number  of  time  step  for  which 
the  global  contact  search  is  valid.  The  output  the  library 
consists  of  the  following.  The  contact  segment  and  node 
pair  list  in  the  processor  in  which  contact  segment  resides. 
The  node  information  consists  of  the  processor  or  sub- 
domain  to  which  it  belongs,  node  co-ordinates  and  the 
velocities.  From  this  point  onwards  each  processors  can 
perform  the  local  contact  search. 

4.3  Parallel  contact  interface  solver 

Once  the  contact  segment-node  pairs  are  found,  the 
normal  and  the  tangential  force  at  the  actual  contact 
surface  between  various  contacting  bodies  needs  to  be 
solved.  For  this  purpose,  we  employ  conjugate  projected 
gradient  solver  developed  in  Sha  et.  al  1996  is  employed. 
The  parallel  implementation  of  this  solver  involves  local 
matrix-vector  operation  and  vector  operations.  In 
addition,  there  are  nearest  neighbor  communications  and 
two  collective  communications.  Since,  predominantly  for 
most  time  of  the  simulation  actual  contact  surface  owned 
by  number  of  processors  are  few  compared  to  the  number 
of  processors  employed  in  the  simulation,  the 
communication  overhead  is  small  especially  for  the 
collective  communication.  Thus,  parallel  overhead 
associated  with  parallel  contact  interface  solver  kernel  is 
very  small  ensuring  overall  parallel  scalability  of  the 
simulation. 

5.  RESULTS 

In  this  section,  illustrative  examples  are  considered 
that  validate  and  demonstrate  the  capabilities  of  the 
proposed  approaches.  In  particular,  the  computational 
CPU  time  savings  by  employing  multi-scale  in  time 
approaches  for  large-scale  computations  such  as  an  entire 
Army  vehicle  simulation  and  the  parallel  performance 
results  are  presented.  The  simulated  results  are  compared 
with  experimental  results  for  validation  purpose  for 
simple  cases. 

An  adaptive  algorithm  based  on  a  smart  switch 
formulation  was  developed,  implemented,  and  tested  for 
various  applications  involving  contact-impact  events. 

First,  the  standard  Taylor  bar  impact  problem  was 
considered  to  compare  the  explicit  only  computations  and 
the  smart  switch  based  explicit/semi-implicit  computation 
with  experimental  results  available  in  the  literature 
(Hanson  and  Hemez,  2002).  The  bar  impacting  a  rigid 
plate  at  245.7  m/sec  was  considered.  The  initial  length  to 
final  length  of  the  bar  and  initial  radius  to  the  final  radius 
of  the  bar  at  the  impacting  surface  of  the  simulation 
results  to  experimental  results  are  compared.  A  finite 
element  mesh  consisting  of  83k  tetrahedral  elements  is 
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Figure  2:  Taylor  bar  impact  problem  with/without  switch 
algorithm  showing  plastic  strain,  and  final  deformed 
shape  of  the  bar  in  comparison  with  experimental  results. 


considered.  The  Fig.  2  shows  the  simulated  results  in 
comparison  with  the  experimental  results.  The  following 
observations  are  evident  from  the  results.  First,  from  the 
plastic  strain  contour  plots,  the  explicit  only  computations 
results  are  similar  to  the  results  of  explicit/semi-implicit 
computations  with  a  smart  switch.  The  final  deformed 
shape  is  in  close  agreement  to  the  experimental  results 
with  an  error  of  2%  on  the  length  and  an  error  of  3%  on 
the  radius.  A  computational  speedup  of  3.98  (CPU 
timings  are  normalized)  is  observed  with  the  smart  switch 
based  computations. 


In  the  second  example,  some  relevant  to  the  parallel 
performance  results,  we  will  consider  again  the  Taylor  bar 
impact  problem.  A  finite  element  mesh  consisting  of  250k 
elements  is  considered.  Figure  3  demonstrates  the  parallel 
speed-up  for  the  same. 


The  third  example  consists  of  ten  spherical  balls 
impacting  a  plate.  This  example  is  considered  to 
demonstrate  the  CPU  advantage  of  the  smart  switch  based 
explicit/semi-implicit  computations  when  multiple 
impacts  are  involved  in  a  random  sequence.  The  Fig  4 
shows  the  stress  contour  plots  of  the  sequence  of  contact- 
impact  events  occurring  when  these  balls  are  impacting 


Figure  3:  Initial  parallel  performance  results  for 
contact-impact  problem  -  Taylor  bar  impact  problem. 

on  the  deformable  plate.  The  bar  chart  demonstrates  that 
CPU  advantage  of  switch  based  computations  of  3.5  times 
over  the  explicit  only  computations. 

Finally,  example  demonstrates  the  large-scale 
computational  capabilities  of  the  software  pCIA-3D.  The 
application  of  this  technology  to  the  initial  FCS  design 
concepts  is  illustrated  in  Fig  5.  The  Fig  5a  shows  the 
entire  Army  vehicle  modeled  with  1.27  million  tetrahedral 
elements  impacted  with  several  projectiles  at  different 
rates  and  different  times  to  simulate  multiple  short-scale 
local  events  and  subsequently  the  long-scale  events  such 
as  vehicle  body  vibrations.  The  goal  here  is  to 
demonstrate  the  prediction  of  the  global  response 
including  structural  vibration  and  wave  propagation  in  the 
structure  for  local  contact  and  impact  events.  The  Fig.  5b 
shows  the  finite  element  mesh  partitioned  into  64 
partitions.  The  Fig.  5c  shows  the  typical  stress  contour  in 
the  structure  for  a  typical  instant  of  time.  The  simulation 
analysis  shows  an  order  of  magnitude  of  saving  in  run 
times  (see  Fig  5d)  as  compared  to  the  standalone  explicit- 
based  computations. 


6.  CONCLUSIONS 

In  this  paper,  we  present  a  new  and  advanced  HPC 
based  multi-scale  in  time  projectile-target  simulation 
environment  for  lethality  and  survivability  that  is  useful 
for  FCS  design  concepts.  Illustrative  examples  were 
considered  that  demonstrate  the  capabilities  of  the 
proposed  approach  to  include:  1)  reduced  run  times  of  an 
order  of  magnitude  for  problems  involving  contact-impact 
and  long-term  post  dynamics  as  against  employing 


Figure  4.  Sequence  of  contact-impact  simulation  of  ten  spheres  colliding  with  each  other  and  a  deformable  plate.  The 
bar  char  illustrates  the  speedup  of  the  simulation  of  3.5  times  of  switching  based  simulation  to  only  explicit  based 
computations. 


standalone  explicit  computation  based  codes,  2)  a  unified 
code  instead  of  a  combination  of  codes  for  avoiding 
erroneous  approximations  introduced  in  combining 
different  codes  and  for  ease  of  maintainability  especially 
in  the  context  of  employing  HPC,  and  3)  better  parallel 
scalability  which  implies  effective  utilization  of  the  Army 
HPC  resources. 
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